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Abstract-Biometric system has been actively emerging in various 
industries for the past few years and it is continuing to roll to 
provide higher security features for access control system. In this 
paper, we propose a multi-modal system for person identification 
using multi-spectral palmprint images. In this work, the 
observation vectors are extracted using 2D Gabor filters. Thus, 
the images {RED, BLUE, GREEN and Nearest-Infrared (NIR)} 
are filtered by a 2D Gabor filter with different orientations and 
then compressed using the PCA. Subsequently, we use the GMM 
and HMM for modeling the observation vector of each band. 
Finally, log-likelihood scores are used for palmprint matching. 
The multi-modal systems fuse information from several modalities 
in order to achieve better identification performance. Therefore, 
all bands are integrated using the fusion at the matching score 
level in order to construct an efficient multi-modal identification 
system. The experimental results showed that the designed system 
achieves an excellent identification rate and provide more security 
than uni-modal system. 

Keywords-Biometrics; Identification; Multi-spectral Palmprint; 
Gabor Filter; PCA; GMM; HMM; Data Fusion 

I. INTRODUCTION 

Automatic personal identification with the use of 
biometrics technologies has been an active research area in 
pattern recognition and computer vision in recent years. In 
addition to the importance of pure research, it has a number of 
field applications such as surveillance and security systems, 
physical buildings and more applications. These technologies 
make use of the physical and behavioural characteristics of 
person such as fingerprint, iris, face, palmprint, signature, and 
voice for personal identification, which can provide advantages 
over non-biometric methods such as PIN, and ID cards [1]. 
Among these biometric technologies, palmprint identification 
is one of the most stable and reliable system. Palmprint 
identification is a biometric technology which recognizes a 
person based on his/her palmprint pattern. Palmprint serves as 
a reliable human identifier because the print patterns cannot be 
duplicated. Compared with other physical biometric 
characteristics, palmprint identification has several advantages 
such as low-resolution imaging, stable line features, and low- 
cost capturing device [2]. It covers wider area than fingerprint 
and it contains abundance of useful information for 
identification. 

In this paper, we first propose a multi- spectral palmprint 
identification algorithm based on single band. In this study, 
feature extraction module was made by using the Gabor phase 
response. During feature extraction stage, five orientations for 



Gabor filters were used. As next step, Principal Components 
Analysis (PCA) was applied to these features. In the modeling 
stage, Hidden Markov Model (HMM) was used. To compute 
the correct identification rate of proposed HMM classifier, it 
was compared to Gaussian Mixture Model (GMM) by using a 
data set containing 300 samples. However, uni-modal systems 
are not perfect and problems like noise in the sensed biometric 
data, non-universality, and the lack of distinctiveness of the 
chosen biometric trait lead to unacceptable error rates in 
recognizing a person. Some of the limitations imposed by 
single modal biometric systems can be overcome by using 
multiple biometric modalities [3]. The multimodal biometric 
systems are expected to be more reliable due to the presence of 
multiple templates security. For that, the paper presents a 
section for fusing information from palmprint images captured 
under different light spectrum (different band) at the matching 
score level. 

The rest of the paper is organized as follows: The proposed 
uni-modal identification scheme is presented in Section II. 
Section III gives a brief description of the region of interest 
extraction. The feature extraction and modeling process, 
including an overview of the Gabor filter, HMM and GMM, is 
presented in Section IV. The obtained results, prior to fusion 
and after fusion, are evaluated in Section V. Finally, 
conclusions and future work are given. 

II. UNI-MODAL SYSTEM DESCRIPTION 

The proposed system is composed of two or more sub- 
systems exchanging information in matching score level. Each 
sub-system exploits different modalities (band images). Each 
uni-modal biometric system (for example, Fig. 1) shows a uni- 
modal biometric identification system based on RED modality. 
This system performs on each band image of the palm surface 
of the hand. In the pre-processing module, the band image is 
segmented using a segmentation procedure described in [4]. In 
the next module, firstly the observation vectors describing the 
shape of the band are computed using Gabor phase response, 
and are compressed using PCA technique. Afterwards, the 
observation vectors are modelled and these models are then 
stored in the system database. In the matching module, the 
matching between the observation vectors and the models 
previously stored in the database is performed using log- 
likelihood score. Finally, in the decision module, the output of 
the matching module is compared with the security threshold, 
and the final decision is made. 
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Fig. 1 An illustration of a typical uni-modal palmprint identification system based on RED band modality 
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Fig. 2 Various steps in a typical region of interest extraction algorithm: (a) the filtered image; (b) the binary image; (c) the boundaries of the binary image and the 
points for locating the ROI pattern; (d) the central portion localization; (e) the pre-processed result (ROI) 



III. PALMPRINT PRE-PROCESSING 



In order to localize the palm area, the first step is to 
preprocess the palm images. We use the preprocessing 
technique described in [4] to align the palmprints. In this 
technique, Gaussian smoothing filter is used to smoothen the 
image before extracting the Region of Interest (ROI) and its 
features. After that, Otsu's thresholding is used for binarization 
of the hand. A contour-following algorithm is used to extract 
the hand contour. The tangent of the two stable points on the 
hand contour (they are between the forefinger and the middle 
finger and between the ring finger and the little finger) are 
computed and used to align the palmprint. The central part of 
the image, which is 128x128, is then cropped to represent the 
whole palmprint. Fig. 2(see above) shows the palmprint pre- 
processing steps. 

IV. FEATURE EXTRACTION AND MODELING 

The feature extraction module processes the acquired 
biometric data and extracts only the salient information to form 
a new representation of the data. Ideally, this new 
representation should be unique for each person. In our method, 
the band modality is typically analysed using Gabor filter and a 
PCA technique. After the decomposition transform of the 
Gabor phase response, some of vector components are selected 
to construct observation vectors. 

A. Observation Vector Generation 

Gabor filters can be used to extract components 
corresponding to different scales and orientations from images. 
The 2D Gabor filter can be represented as a complex sinusoidal 
signal modulated by a Gaussian function (envelope). 
Specifically, a 2D Gabor filter h8(x, y) can be formulated as 
follows [5]: 



h g (x, y) = 



1 



_g((-r"+y )/2a ) ^2mu(xcos6+ysm6 



(1) 



Where 1 = , u is the frequency of the sinusoidal signal, 
8 controls the orientation of the function, and o is the standard 
deviation of the Gaussian envelope. In order to select Gabor 
filters for band pass filtering, three parameters have to be 
determined: frequency u, orientation 8, and space constants o. 
The values of 8 only in the interval [-90o, 90o] are considered, 
since other values are redundant due to symmetry. 

In this work, the parameters of Gabor filters were 
empirically determined for the acquired palmprint band images. 
These were set as; u = 0.0916, and o = 5.6179. Gabor filters 
with five different values of 8 (-60o,-30o, Oo, 30o, 60o) were 
employed. Filtering the image I(x, y) with the Gabor filter h8 
(x, y), can be defined by the following equation: 



Ig{x, y) = h (x,y)* I(x,y) 



h g (m, ri)I(x — m, y — n) 



(2) 



Where * denotes discrete convolution and the Gabor filter 
mask is of size NxN, with N = 16. Thus every image is filtered 
with a bank of five filters to generate five filtered images. The 
results Re{18}, and Im{16} of a pair of a real and an imaginary 
filter are combined in the Gabor phase response, v|/6 as follows: 



y/ = arctan 



(3) 



The features are generated from \|/8 by PCA technique [6]. 
This feature-extraction technique has been widely used for 
pattern recognition, as well as in the field of biometrics. 
Applying the PCA technique to the vectors v|/6 for each filtered 
images (v|/8), we decorrelate the vectors of v|/6, and concentrate 
the information content on the first components of the 
transformed vectors. The observation vector is formed by 
taking some components of the transformed vectors (for each 
orientation, 0) and concatenating all transformed vectors on 
one vector (observation vector). Fig. 3 (see above) illustrates 
the proposed feature extracted method. 
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Fig. 3 The observation vector extraction algorithm 



B. Modeling Process 

1 ) Gaussian Mixture Model: 

Gaussian mixture model is a pattern recognition technique 
that uses an approach of the statistical methods [7]. The 
observation vector of each class measurement can be described 
by normal distribution, also called Gaussian distribution. Each 
class measurement may be then defined by two parameters: 
mean (average) and standard deviation (variability). Suppose 
that the observation vector is the discrete random variable 
VObs. For the general case, where vector is multidimensional, 
the probability density function of the normal distribution is a 
Gaussian function: 



1 



-exp 



<y*.-Mfv 1 <y*.-M) 



(4) 



Where u. is the mean, X is the co variance matrix and d is the 
dimension of feature vector. Covariance matrix is the natural 
generalization to higher dimensions of the concept of the 
variance of a random variable. If we suppose the random 
variable measurement is not characterized only with simple 
Gaussian distribution, we can then define it with multiple 
Gaussian components. GMM is a probability distribution that 
is a convex combination of other Gaussian distributions: 



(5) 



Where M is the number of Gaussian mixtures and 7ii is the 
weight of each of the mixture. After GMM is trained, the 
model of each user will be the final values of Jti, pi and Si. 
Thus, the compact notation 0, such fhat# = ^ , is 

used to represent a model. To estimate the density parameters 
of a GMM statistic model, Expectation-Maximization 
algorithm (EM) is adopted [8]. 

2) Hidden Markov Model: 

A hidden Markov model is a collection of finite states 
connected by transitions. Each state is characterized by two 
sets of probabilities: a transition probability and either a 
discrete output probability distribution or continuous output 
probability density function which, given the state, defines the 
condition probability of emitting each output symbol from a 
finite alphabet or a continuous random vector [9]. An HMM 
can be written in a compact notation X= (A, B, ti) to represent 
the complete parameter set of the model, where A is the state 
transition probability distribution, B is probability distribution 



of observation symbols for each state, and ji is the initial state 
distribution of any given state. Finally, forward backward 
recursive algorithm, Baum- Welch or otherwise EM algorithm, 
and Viterbi algorithm are used to solve evaluating, training, 
and decoding, respectively [10]. 

V. EXPERIMENT RESULTS AND DISCUSSIONS 

A. Experimental Database 

Experiments are performed on the multi-spectral palm print 
database from the Hong Kong polytechnic university (PolyU) 
[11]. The database contains images captured with visible and 
infrared light. Multi-spectral palmprint images were collected 
from 300 volunteers, including 195 males and 105 females. 
The age distribution is from twenty to sixty years old. It has a 
total of 14400 images obtained from about 300 different palms. 
These samples were collected in two separate sessions. In each 
session, the subject was asked to provide six images for each 
palm. Therefore, twenty-four images of each illumination from 
two palms were collected from each subject. The average time 
interval between the first and the second sessions was about 
nine days. 

B. GMM-based Identification 

1 ) Open Set Identification: 

When an unknown sample is presented to the identification 
system, the system generates an observation vector and 
calculates the Log-likelihood score of this vector given each 
model stored in the database. Thus, a Min-Max normalization 
scheme was employed to transform the Log-likelihood scores 
computed into similarity scores in the same range, [0...1]. In 
this section, the proposed method was tested through the open- 
set identification. In all experiments, the impostor and genuine 
distributions are generated by 2700 and 403650 comparisons, 
respectively. All the Experiments are performed on the multi- 
spectral palmprint (MSP) database from Hong Kong 
Polytechnic University. 

aj Number of Gaussian in the GMM: A series of 
experiments were carried out using the MSP database to select 
the best number of Gaussian in GMM, and this is carried out 
by comparing all k Gaussian, k = 1 to 5, and finding the 
number of Gaussian in GMM that gives the best identification 
rate. The problem we address is as follows: we want the chosen 
number of Gaussian in GMM such that the Equal Error Rate 
(EER) is minimized. In Fig. 4. (a), we compare the system 
performance under different number of Gaussian in GMM. The 
results show the benefits of using one Gaussian in GMM. 
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b) Single band palm-print image: The goal of this 
experiment was to evaluate the system performance when we 
using information from each bands. Fig. 4.(b) compares the 
performance of the system for deferent bands. It can safely be 
seen the benefits of using the BLUE band than the other bands 
in terms of EER. It can achieve an EER equal to 0.327 % at the 
threshold To = 0.8035. Therefore, the system can achieve 
higher accuracy at the BLUE band when compared with the 
other bands. Finally, the ROC curve obtained by the proposed 
scheme is plotted in Fig. 4.(c). Table I shows the False 
Acceptance Rate (FAR) and False Rejection Rate (FRR) with 
percentage using all bands at deferent thresholds. 

c) Multiple band palmprint image: At the matching score 
level fusion, the matching scores output by multiple matchers 
(subsystem) are integrated. In our system, two combinations of 
bands (RGB and RGBN) and different fusion rules, such as 
Sum-score (SUM), Min-score (MUST), Max-score (MAX), Mul- 
score (MUL) and Sum-weighting score (WHT), were tested to 
find the combination that optimizes the system accuracy. Thus, 
to find the better of the all fusion rules and combinations, with 
the lowest EER, Table II tabulates EER for the two 
combinations and fusion rules. As can be seen, the best result 
was obtained with the combination of RGBN and the fusion 
rule was MUL rule, it can achieve even higher precision, an 
EER of 0.005 % and a To of 0.6184. 

2) Closed-set Identification: 

In the case of a closed set identification, a series of 
experiments were carried out to select the best color band (best 
combination). 

a) Single band palm-print image: For the evaluation of the 
closed set identification, Table III presents the experiments 
results obtained for all color bands. From this Table, it can be 
seen that the GREEN band done the best performance. Thus, 
the performance of the closed set identification system is 
significantly improved by using this band. The Rank-One 
Recognition (ROR) rate = 97.556 % with a lowest Rank of 
Perfect Recognition (RPR) of 1 10. 

b) Multiple band palmprint images: A performance 
comparison of all fusion rules of all bands combination (RGB 
and RGBN) is made in Table IV. This table shows that the 
SUM rule with RGBN combination offers better results in 
terms of the ROR (ROR = 99.815 % with RPR = 8). The 
results suggests that always SUM rule has performed better 
than other fusion rules, in the case of RGB combination (ROR 
= 99.667 % with RPR = 9). 

3) Summary Study: 

According to the experimental results, multimodal 
biometrics performs better than the individual biometrics in 
both cases closed set and open set identification (see Fig. 5. (a) 
and Fig. 5.(b). This shows that multimodal biometrics have the 
ability to improve the identification accuracy over individual 
biometrics. This result is shown clearly at the case of RGBN 
combination. 

C. HMM-based Identification 



1 ) Open-set Identification: 

In this section, the results for our proposed method were 
obtained using the HMM modeling. In order to evaluate the 
benefit of the HMM modeling, we perform just as the 
experiments presented in the previous section (GMM 
modeling). 

a) Number of states in the HMM: The experimental setup 
is similar to that described in the first part of this section. 
However, the following gives the results of our method as a 
ROC curves, which shows the FAR and FRR at each threshold 
value. Fig. 6.(a) shows ROC curves generated by five states of 
HMM, from which we can see that two states perform the best 
among the others evaluated in terms of EER. Note that the 
number of Gaussian in each state is equal to 1 . 

b) Single band palmprint image: In this section, we 
compare the performance of all bands. Fig. 6.(b) compares the 
performance of the system for deferent bands. The 
experimental results indicate that the BLUE band performs 
better than the others (EER = 0.377 % with To = 0.7878). 
Finally, the ROC curve obtained by the proposed scheme is 
plotted in Fig. 6.(c). Table V provides the performance of the 
system under all bands. 

c) Multiple band palmprint images: To find the better 
fusion rules and the best combination, Table VI is generated. 
This table shows that the SUM, WHT, and MUL rule offers 
better results in terms of the EER (EER = 0.037 % with To = 
0.7670, To = 0.7789, and To = 0.4425, respectively). Finally, 
the results show the benefits of using the GMM modeling in 
the uni-modal and multi-modal system. 

2) Closed Set Identification: 

In this section, we compare the performance of all bands in 
the case of closed set identification. 

a) Single band palmprint image: In this case, a series of 
experiments were carried out to select the best palmprint band. 
This has been done by comparing all bands and finding the 
band that gives the best identification rate. Table VII shows the 
results for all bands. From this table, it is clear that our system 
achieves a best performance when using BLUE band (ROR = 
97.296 % and RPR = 142). 

b) Multiple band palmprint images: Table VIII shows 
the close set identification results of the multimodal system. 
This Table shows that MUL fusion rule and RGBN 
combination performs better than the other rules and improves 
the original performance by ROR = 99.815 % and RPR = 32. 
Also, we can see that SUM rule followed by WHT rule fusion 
can produce a best performance, ROR equal to 99.778 % (RPR 
= 38) and 99.741 % (RPR = 42), respectively. 

3) Summary Study: 

Fig. 7. (a) and Fig. 7.(b) show the comparison of uni-modal 
and multimodal systems in the closed/open set identification, 
respectively. Results show that the identification rate of 
multimodal identification based on fusion of all bands is higher 
than that of the identification adopting the uni-modality. 
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Fig. 4 GMM-based uni-modal biometric identification test results: (a) the ROC curves under different number of Gaussian; (b) the ROC curves for all palm-print 

bands and f c) the ROC curve for the blue band 



TABLE I GMM-BASED OPEN SET UNI-MODAL SYSTEMS PERFORMANCES 



DB SIZE 




BLUE 






GREEN 






RED 






NIR 




T 


FAR 


FRR 




FAR 


FRR 


T 


FAR 


FRR 


T 


FAR 


FRR 




0.6700 


2.121 


0.000 


0.6200 


3.804 


0.000 


0.6100 


4.882 


0.037 


0.6200 


5.207 


0.222 


300 


0.8035 


0.327 


0.327 


0.7856 


0.370 


0.370 


0.7870 


0.482 


0.482 


0.7858 


0.926 


0.926 




1.0000 


0.007 


2.704 


1.0000 


0.008 


2.444 


1.0000 


0.011 


3.778 


1.0000 


0.021 


0.889 



TABLE II GMM-BASED OPEN SET MULTI-MODAL SYSTEMS PERFORMANCES 



COMBINATION 


SUM 




WHT 


MIN 


MAX 


MUL 


T 


EER 


T EER 


T„ EER 


T EER 


T EER 


RGB 


0.8184 


0.029 


0.8331 0.026 


0.7670 0.098 


0.8764 0.185 


0.6169 0.024 


RGBN 


0.8614 


0.006 


0.8438 0.010 


0.7461 0.100 


0.9243 0.148 


0.6184 0.005 



TABLE III GMM-BASED CLOSED SET UNI-MODAL SYSTEMS PERFORMANCES 



„™ BLUE GREEN RED NIR 

Do olZiii 

ROR RPR ROR RPR ROR RPR ROR RPR 

300 97.296 41 97.556 110 96.222 251 94.111 201 



TABLE IV GMM-BASED CLOSED SET MULTI- MODAL SYSTEMS PERFORMANCES 



COMBINATION 


SUM 


WHT 


MIN 


MAX 


MUL 


ROR RPR 


ROR RPR 


ROR RPR 


ROR RPR 


ROR RPR 


RGB 


99.667 9 


99.556 11 


99.370 99 


96.222 38 


99.667 10 


RGBN 


99.815 8 


99.778 10 


99.407 99 


94.407 41 


99.815 9 



1K 



S 98.5 

1 BS 
1 



67.5 



E.JE 

RGB (MUL) 

— RGBN 'MUL) 



Iff* Id 3 Iff 1 Id" Iff 5 10" IE? 
FalsE accept rale \W\ 



ICC 



b=.5 



3 

9E 



B7.5 




1D 2D 3D 4D5D 6D 



- BLUE 

- RGB (SUM) 
-RGBN^SUM) 



5D 1DD 153 EC IcO 3DD 
Rank 



Fig. 5 Comparison of different GMM based systems: (a) open set identification systems and (b) closed set identification systems 
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Fig. 6 HMM-based uni-biometric identification test results: (a) the ROC curves under different number of states; (b) the ROC curves for all palm-print bands; (c) 

the ROC curve for the blue band 



TABLE V HMM-BASED OPEN SET UNI- MODAL SYSTEMS PERFORMANCES 





BLUE 






GREEN 




RED 






MR 




DB SIZE 

T 


FAR 


FRR 


T 


FAR 


FRR 


To 


FAR 


FRR 


To 


FAR 


FRR 


0.6000 


4.153 


0.074 


0.6100 


3.687 


0.000 


0.6300 


2.850 


0.037 


0.6500 


4.528 


0.296 


300 0.7878 


0.377 


0.377 


0.7655 


0.493 


0.493 


0.7624 


0.523 


0.523 


0.7794 


0.919 


0.919 


1.0000 


0.008 


2.704 


1.0000 


0.010 


3.000 


1.0000 


0.011 


3.741 


1.0000 


0.020 


5.963 


TABLE VI HMM-BASED OPEN SET MULTI-MODAL SYSTEMS PERFORMANCES 


COMBINATION 




SUM 




WHT 




MIN 




MAX 


MUL 


T 


EER 




To 


EER 


To 


EER 


T„ 


EER 


To 


EER 


RGB 
RGBN 


0.7495 0.105 
0.7670 0.037 


0.7608 
0.7789 


0.099 
0.037 


0.7618 
0.7577 


0.111 

0.096 


0.8878 
0.9187 


0.148 
0.148 


0.4885 
0.4425 


0.098 
0.037 



TABLE VII HMM-BASED CLOSED SET UNI- MODAL SYSTEMS PERFORMANCES 



DB SIZE 


BLUE 




GREEN 


RED 






NTR 


ROR 


RPR 




ROR RPR 


ROR RPR 




ROR 


RPR 


300 97.296 


142 




97.000 126 


96.259 239 




94.037 


211 


TABLE VIII HMM-BASED CLOSED SET MULTI- MODAL SYSTEMS PERFORMANCES 


COMBINATION 


SUM 




WHT 


MIN MAX 






MUL 


ROR 


RPR 


ROR RPR 


ROR RPR ROR 


RPR 




ROR RPR 


RGB 


99.630 


44 


99.556 48 


99.000 131 95.963 


66 




99.630 40 


RGBN 


99.778 


38 


99.741 42 


99.185 110 93.963 
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Fig. 7 Comparison of different HMM based systems: (a) open set identification systems and (b) closed set identification systems 
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Fig. 8 Comparison of the GMM-based and HMM-based systems: (a) open set identification systems and (b) closed set identification systems 

[7] Peter Varchol, Dusan Levicky, "Using of Hand Geometry in Biometric 
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Processing, vol. 56, issue 7B, pp. 2797-281 1, 2008. 



D. Comparison Study 

To find the better methods, GMM modeling or HMM 
modeling, graphs showing the ROC curves for the closed set 
and open set identification using uni-modal and multimodal 
system, were generated (see Fig. 8. (a) and Fig. 8.(b)). By the 
analysis of those plots, it can be observed that the performance 
of the identification system is significantly improved by using 
the fusion. In the case of open set identification, the GMM 
modeling gives the best result (EER = 0.005 %). The best 
result (ROR = 99.8150) in the case of closed set identification 
is given when using HMM modeling. However, it can be 
concluded that the fusion of all modalities yields much better 
identification results compared with uni-modality. Therefore, 
the developed system is expected to give higher accuracy. 

VI. CONCLUSION AND FURTHER WORK 

This paper proposes an efficiency scheme for MSP 
identification using the GMM (HMM) modeling. Firstly the 
observation vector is given by Gabor phase response with 
different orientations and compressed using PCA. 
Subsequently, we use the GMM (HMM) for modeling the 
observation vector of each band. The experimental results 
showed that the proposed methods obtained a highest 
recognition rate and the information fusion at the matching 
scores level improved the results. For further improvement of 
the system, our future work will focus on the performance 
evaluation using large size database and using other feature 
extraction techniques such as Independent Component 
Analysis (ICA). 
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